Emergence of spontaneous anticipatory hand movements in a probabilistic environment

In this article, we present a novel experimental approach to the study of anticipation in probabilistic cuing. We implemented a modified spatial cuing task in which participants made an anticipatory hand movement toward one of two probabilistic targets while the (x, y)-computer mouse coordinates of their hand movements were sampled. This approach allowed us to tap into anticipatory processes as they occurred, rather than just measuring their behavioral outcome through reaction time to the target. In different conditions, we varied the participants’ degree of certainty of the upcoming target position with probabilistic pre-cues. We found that participants initiated spontaneous anticipatory hand movements in all conditions, even when they had no information on the position of the upcoming target. However, participants’ hand position immediately before the target was affected by the degree of certainty concerning the target’s position. This modulation of anticipatory hand movements emerged rapidly in most participants as they encountered a constant probabilistic relation between a cue and an upcoming target position over the course of the experiment. Finally, we found individual differences in the way anticipatory behavior was modulated with an uncertain/neutral cue. Implications of these findings for probabilistic spatial cuing are discussed.

Emergence of spontaneous anticipatory hand movements in a probabilistic environment Pernille Bruhn department of Psychology, Aarhus University, denmark anticipation, prediction, probabilistic spatial cuing, statistical learning, computer mouse tracking in this article, we present a novel experimental approach to the study of anticipation in probabilistic cuing. We implemented a modified spatial cuing task in which participants made an anticipatory hand movement toward one of two probabilistic targets while the (x, y)-computer mouse coordinates of their hand movements were sampled. this approach allowed us to tap into anticipatory processes as they occurred, rather than just measuring their behavioral outcome through reaction time to the target. in different conditions, we varied the participants' degree of certainty of the upcoming target position with probabilistic pre-cues. We found that participants initiated spontaneous anticipatory hand movements in all conditions, even when they had no information on the position of the upcoming target. however, participants' hand position immediately before the target was affected by the degree of certainty concerning the target's position. this modulation of anticipatory hand movements emerged rapidly in most participants as they encountered a constant probabilistic relation between a cue and an upcoming target position over the course of the experiment. Finally, we found individual differences in the way anticipatory behavior was modulated with an uncertain/neutral cue. implications of these findings for probabilistic spatial cuing are discussed. a specific sequence of events (e.g., of the type "A → B") then event A will eventually acquire signal value for the future event B, and the organism will start to anticipate B when it encounters A (i.e., before B actually occurs). However, the relationship between a given signaling event (A) and the signaled upcoming event (B) is not always a perfect causal relationship, that is, the predictive relation between many natural events is probabilistic rather than deterministic (e.g., Feigenberg, 1969). Thus, for anticipation to be functional in a nondeterministic world, the probability of a future event should be taken into account and shape anticipatory processes in a systematic way.
The present study investigated how repeated experience with probabilistic sequences of events shapes anticipatory processes. In our experimental task, participants moved the computer mouse from an initial position at the bottom of the computer screen to click on a target that would occur either on the left or on the right upper part of the screen. (This basic setup was inspired by previous studies which introduced the use of mouse tracking in cognitive tasks; see McKinstry, Dale, & Spivey, 2008;Spivey, Grosjean, & Knoblich, 2005.) Before the presentation of the target, one of five possible cues was presented centrally on the screen. Unbeknownst to the participants, each of these cues represented a given conditional probability of the target subsequently occurring on the left or on the right side of the screen. Hence, each cue contained information on how certain the participants could be about the position of the upcoming target on the current trial. This degree of certainty concerning the upcoming target's position was varied over three mixed conditions: (a) a certain condition (100%-valid cue), (b) a semi-certain condition (75%-valid cue), and (c) an uncertain condition (neutral/uninformative cue). The specific probabilistic relation between a given cue and a given subsequent target position was not given explicitly to the participants. However, we expected that the participants would gradually "pick up" these probabilistic regularities and that this would become reflected in their anticipatory hand movements toward the targets. Studies on statistical learning have shown that even infants are able to learn the implicit probabilistic structure of sequences (of, e.g., different visual forms) and the spatiotemporal occurrence of such forms (Kirkham, Slemmer, & Johnson, 2002;Kirkham, Slemmer, Richardson, & Johnson, 2007). In our study, we were not concerned with whether participants' (potential) "learning" of the probabilistic predictive relationship between cues and targets was implicit or explicit (see Dale, Duran, & Morehead, 2012, for a discussion of implicit and explicit learning related to anticipatory behavior in a statistical learning task). Simply, we wanted to explore if, how quickly, and in which way emerging probability-based degrees of certainty modulate anticipatory behavior.
Because anticipation by definition takes place before the event it is directed at, traditional outcome-based measures such as reaction time and accuracy only offer an indirect, post hoc measure of anticipation.
By measuring participants' hand movements before the target occurred we were able to tap into anticipatory processes as they occurred, rather than just indirectly measuring anticipation through target responses.

Participants
Eleven undergraduate students (seven females and four males between 20 and 22 years old, M age = 21.2, SD = 0.7) from the Department of Psychology, Aarhus University participated as unpaid volunteers. Data from two of the 11 participants were excluded from the analysis as these participants reported that they had voluntarily ignored the cues throughout the experiment.

Apparatus
The experiment was programmed and run using E-Prime (Psychology Software Tools, Inc., Pittsburgh, PA). Stimuli were presented on a 15" computer monitor (800 × 600 pixels), and responses were given by moving a standard computer mouse and clicking its left button. The mouse cursor settings used were Windows' standard settings.

Experimental set-up and stimuli
An example of a single trial is shown in Figure 1. At the beginning of each trial, a start box was displayed at the bottom of the screen. The mouse cursor was visible and locked to a position in the middle of the start box. Thus, the initial position of the mouse cursor was the same on every trial within and across participants. The participant initiated the trial by clicking the left mouse button, after which one out of five possible cues appeared at the center of the screen. Simultaneously, the mouse cursor was unlocked allowing the participant to move the mouse cursor on the screen. After a 680-ms interval, the target was displayed at the left or right upper part of the screen. This provided ample time for the participant to initiate anticipatory mouse movements before the target occurred. Finally, the participant clicked on the target and the trial was terminated. A blank screen was displayed for 1,500 ms before the onset of the next trial (see Figure 1).
The five different cues and their corresponding probabilities that the subsequent target would occur on the left versus the right side of the screen are shown in Figure 2. Rather than using arbitrary symbolic cues, we chose cues whose physical appearance was designed to intuitively make sense as a visual signal for their specific predictive value regarding the upcoming target position. This was done because we wanted to make it moderately easy for the participants to develop a sense of the different cues' predictive value, allowing for a potential effect of the degree of certainty to develop over the course of the experimental session.

Instructions
At the beginning of the experimental session, instructions were given to the participants via an instruction screen on the computer monitor, which contained both text and figures. The participants were told to read the instructions thoroughly before starting. The instructions screen stated that the experiment investigated how quickly we respond to objects presented in different parts of the visual field and, further, that the participant should start each trial by clicking on the start box at the bottom of the screen and that, immediately hereafter, one of five possible figures (i.e., the cues) would occur. The five cues were displayed on the instruction screen, but with no reference to their predictive value. This was done in order to ensure that the participants paid attention to the cues during the experiment, while keeping them initially naïve of their different predictive values. 1 Finally, the instructions stated that a green square would occur at the left or right upper part of the screen shortly after the black and white figure and that the participant's task was to click as quickly as possible on the green square.
No further instructions were given.

Data collection and preprocessing
Throughout each trial, the following data were collected: (a) the time of the sequence and positioning of stimuli in a single trial. the start box was a 50 × 30 pixels rectangle with the word "start!" written in green inside. the center of the rectangle was positioned at the horizontal midline, 35 pixels above the bottom of the screen. the cues were 100 × 100 pixels black and white squares positioned with their center at the intersection of the horizontal and the vertical midlines of the screen. the target was a 100 × 100 pixels square with green borders positioned with its center 170 pixels from the upper left or right corner of the screen for the left-and right-side targets, respectively. the relative sizes and positions of the objects in this figure roughly correspond to those of the experiment. the text below each screen diagram indicates the duration of the screen display.

Experimental session
Each participant performed one experimental session of 290 trials, with a total duration of approximately 20 min. The 290 trials were divided into five successive blocks of 58 trials, which were enumerated as follows: the first 58 trials = Block 1, the next 58 trials = Block 2, etc.
As illustrated in Table 1, the experimental structure was constructed to yield an equal number of trials with each of the five cues within each block and within the experiment as a whole. Therefore, there was a higher number of trials in the certain and the semi-certain condition compared to the uncertain condition. The order of the cues (and the associated target positions) was randomized without replacement within each block anew for each participant. Hence, trials from the three degree-of-certainty conditions also occurred in a random order within each block. However, the number of occurrences within each block of the different Cue × Target trial-types was constrained to assure that both the 75:25 ratio of valid versus invalid trials in the semicertain condition and the 50:50 ratio of left versus right target side in the uncertain condition were present within each block (see Figure 2 and Table 1). Hence, the conditional probability of getting an invalid target outcome in the semi-certain condition (i.e., 25%) was the same within a given block as within the experiment as a whole. Likewise, the conditional probability of getting, say, a left target outcome in the uncertain condition (i.e., 50%) was the same within a given block as within the experiment as a whole. This design allowed us to look at the evolution of the effect of degree-of-certainty over the experiment by using Block as a factor (with five levels).

Dependent and independent variables
We wanted to investigate the emergence over time of differences in participants' anticipatory behavior between the different degrees of certainty. Anticipatory behavior was operationalized by taking a "freeze-frame" immediately before the target occurred of the participant's anticipatory hand movement on a given trial. This corresponded to measuring the x-and y-coordinates of the computer mouse 680 ms after cue onset. Hence, we had two dependent variables: the pre-target x-coordinate (the x-coordinate 680 ms after cue onset) and the pre- We used two independent variables for the analyses of this experiment: the block variable (described above) and the degree-of-certainty variable (described below). In this experiment, we were interested in if, and to what extent, participants made anticipatory movements toward the most likely target side (as indicated by the cue), but it was in principle irrelevant whether the cue indicated a high probability on the left or the right side. Thus, for the purpose of analyzing effects on anticipatory mouse movements (i.e., movements occurring before the target) we can consider the following two Cue × Target  Finally, we can consider the following two Cue × Target conditions functionally identical: (a) cue: "50-50"/target: left and (b) cue: "50-50"/ target: right (uncertain condition). Hence, the eight levels of the Cue × Target conditions shown in Figure 2 were coded into the three-level degree-of-certainty variable as depicted in Table 1. Degree-of-certainty reflected how certain the participants could be about the position of the upcoming target before its occurrence, regardless of whether the target was cued to occur on the left or the right side and whether the target actually occurred on the left or the right side. overview of the Block/session structure and the independent variables a The smaller number of trials with the "0-100" cue compared with the other four cues is due to a programming error. This was the case for all participants and did not affect the predictive value of the "0-100" cue (i.e., it was always followed by a right target).
To be sure that it was unproblematic to collapse our anticipatory mouse movement data across upcoming target side, we performed a preliminary analysis comparing the magnitude of the anticipatory movements for left and right targets. We conducted ANOVAs of the effect of target side (left vs. right) on the absolute pre-target x values for each of the three degree-of-certainty conditions separately and found no effect of target side on pre-target  x-values movement toward the right side.

Effect of the degree of certainty on the pre-target hand position
As illustrated by the graphs in Figure 3, it is evident that the effect of the degree of certainty on the pre-target x-and y-coordinate is particularly present from the second block on. Therefore, we used the pooled data

Continuous spatiotemporal characteristics of the anticipatory hand movements
The graphs in Figure 4 suggest the existence of two phases in the participants' anticipatory hand movements occurring from start-click to target occurrence. First, there is an initial upward going movement, which was not modulated by the degree of certainty, starting approximately 200 ms after the cue (see Section C of Figure 4): The

Individual differences in anticipatory hand movements in the uncertain condition
Because of the broad distribution of the pre-target x-coordinates in the uncertain condition ( Figure 3, Section C, bottom row), we decided to look at single participants' contributions to the different ranges of values in this condition. To do this, we created an eccentricity variable using data from Blocks 2-5. We assigned a value of 0 for trials in which the pre-target x-coordinate ranged from -100 to 100 and a "1" for trials in which the pre-target x-coordinate was lower than -100 or higher than 100. Hereafter, we treated eccentricity as a continuous variable

dIscussIon
First of all, our mouse tracking data revealed that participants spontaneously engaged in anticipatory hand movements in all three degreeof-certainty conditions. These spontaneous anticipatory hand movements are evidenced by the anticipatory trajectories (cf. Figure 4) in conjunction with the very low movement initiation times. An interesting finding is that this tendency to spontaneously engage in anticipatory behavior was equally present for the three degrees of certainty. It is evident from the y-coordinate data shown in Section C of Figure 4 that participants moved the mouse upward even in the uncertain condition. Hence, even when participants did not know which side to go to, rather than just awaiting passively to react to the target when it although the overall predictability of a sequence of stimuli was manipulated block-wise in their experiment. Note moreover that classical spatial cuing experiments also typically manipulate cue validity on a trial-to-trial basis, so these are also subject to potential carryover effects.
Whereas the tendency to initially engage in anticipatory hand movements did not differ between the different conditions in our experiment, the probability/degree of certainty concerning the upcoming target position significantly affected the spatial characteristics of the anticipatory hand movements. The plots of the evolution over the course of the experiment of the effect of degree-of-certainty on the pre-target x-and y-coordinates (Figure 3, Sections A and B) reveal that there was a rapid emergence of probability-based modulation of participants' hand position immediately before the target. Hence, presenting participants with a constant probabilistic relationship between given cues and given upcoming target positions resulted in the recurring probability structure of the "experimental world" being rapidly (i.e., within the first 58 trials) assimilated by the participants in a way that caused them to modulate their anticipatory hand movements correspondingly. This rapid emergence of explicit behavioral effects of the exposure to an implicit statistical structure is in line with findings from studies on statistical learning (e.g., Dale et al., 2012;Kirkham et al., 2002Kirkham et al., , 2007. Although the effect of the degree of certainty on participants' pre-target hand position appeared already after the first block and stayed rather stable from the second block, there seemed to be a small change over the course of the experiment in the shape of the underlying distribution of trials. Based upon visual inspection of the distributions of the pre-target x-coordinates in the semi-certain condition (see Figure 3, Section C, second row), there appears to be a gradual evolution from a somewhat bimodal shape of the distribution in Block 2 toward a more unimodal distribution in Block 5. This pattern might suggest that the consolidation over time of probabilitybased anticipation have complex effects on behavior that are not detectable in mean values. However, due to the relatively small number of observations in the Block × Degree-of-certainty subsets of data we cannot conclude statistically on this apparent change in the shape of the distribution.
As depicted in Figure 4 (Section A), we found that when the cue predicted the upcoming target position with 100% certainty, then participants made anticipatory hand movements relatively directly toward the predicted position. This is in contrast to the (averaged) anticipatory movement trajectories observed in the uncertain condition, which stayed in the center with equal distance toward the two targets. When we compare the averaged anticipatory trajectories of the certain and semi-certain conditions, there is a lateral shift toward the alternative/ unlikely target side in the semi-certain relative to the certain condition, reflected in the pre-target x-coordinate being lower in the semi-certain than in the certain condition. This finding of a behavioral difference between the certain and the semi-certain condition is consistent with findings from spatial cuing studies showing that semi-certainty (70 to 90%) yields higher reaction times than complete certainty of outcome (Drazin, 1961;Eriksen & Yeh, 1985;Geng & Behrmann, 2005). In the context of the present study, this lateral shift can be taken as an indica-tion of a higher spatial attraction toward the alternative/unlikely target side in the semi-certain than the certain condition. This finding is in line with a more general picture that has emerged from previous mouse tracking studies: Higher degrees of ambiguity/uncertainty in a dichotomous decision task (McKinstry et al., 2008) and a phonological processing task (Spivey et al., 2005) have been associated with higher spatial inclination of hand movements toward an alternative choice category.
A potential methodological limitation of this study concerns the fact that the cues were not counterbalanced, that is, the darker side of the cue always coincided with the high-probability side in the certain and semi-certain conditions. We cannot exclude that the differences observed in anticipatory hand movements between our conditions were the result of a tendency to automatically move the mouse towards the darker side. However, effects of a degree-of-certainty manipulation on anticipatory hand movements similar to the ones reported here were observed in another study in which cues with a darker and a lighter side were counterbalanced across participants such that the lighter side of the cue corresponded to the high-probability side for half of the participants (Bruhn, Huette, & Spivey, 2012). It is therefore unlikely that the effects observed in the current study were the result of a bias of inherently induced movement towards the darker side.
It has been proposed that the continuous motor output sampled with computer mouse tracking can be conceived of as a two-dimensional projection of the ongoing perceptual and cognitive dynamics involved in accomplishing the current task (Spivey, 2007; see also Freeman, Dale, &Farmer, 2011, andMagnuson, 2005). Hence, the anticipatory hand movements measured in this study can be seen as a reflection of the underlying (psychological and, possibly, neural) anticipatory processes. From this perspective, the higher spatial attraction of the anticipatory hand movements toward the alternative target side in the semi-certain condition could be taken to suggest that when we can only be semi-certain about the outcome of two alternative upcoming events we do not "go" entirely for the most likely event, but rather engage some resources in anticipating the unlikely event also (see , for further support and discussion of this interpretation).
A central issue addressed by previous mouse tracking research has been whether a higher average spatial attraction toward an alternative location is indeed the result of a continuous, graded modulation of spatial attraction on a trial to trial basis, rather than being the result of the presence of a subset of trials with very high spatial attraction toward the alternative side in the high-attraction condition (see e.g., Spivey et al., 2005). In the context of the present study, this latter scenario would be associated with a bimodal distribution of the pre-target x-coordinates in the semi-certain condition. As noted above, such bimodality may be present in (a subset of) the semi-certain trials, but cannot be statistically tested for because of too few observations.

Continuous spatiotemporal characteristics of anticipatory behavior
In addition to the "freeze-frame" of anticipatory hand movements, conveyed through the pre-target x-and y-coordinates, we also pre- Indeed, Gibson and Kingstone (2006) found an effect of four different types of spatial cues on reaction time after only 250 ms, that is, some hundred milliseconds before the effects of the cue was visible in the continuous anticipatory hand movements in our experiment.
Rather, what is interesting about the current results is that the anticipatory hand movements of the first phase (from approx. 200 to 400 ms) indicate the existence of anticipatory processes that are not dependent on the information conveyed by the cue and that are probably initiated already before the cue was displayed. Future studies could further investigate this intriguing finding of individual differences.

Implications for probabilistic spatial cuing
Studies on probabilistic spatial cuing have traditionally used measures of reaction time and accuracy to infer on processing that occurs before these measures are collected (e.g., Downing, 1988;Eriksen & Yeh, 1985;Geng & Behrmann, 2005;Gottlob, Cheal, & Lyon, 1999;Posner, 1980, among many others). The mouse tracking approach used in the present study provided novel insight into anticipatory processes that has implications for probabilistic spatial cuing. Specifically, the findings presented here suggest that: (a) anticipatory processes directed at the upcoming target can be initiated before a cue is presented, (b) the condition typically presumed to be "neutral" in spatial cuing paradigms (i.e., the condition where no information on the location of the upcoming target is given, here referred to as the uncertain condition) can induce unique anticipatory processing of its own, and (c) as also supported by reaction time studies, a "valid cue" cannot necessarily be considered a unitary concept that can be unproblematically compared across different (high) degrees of certainty because conditions of 100% certainty (i.e., using a 100%-valid cue) and semi-certainty (using a generally valid cue) appears to engender differential anticipatory processes.

Anticipation and computer mouse tracking
The interest for understanding anticipatory/predictive processes in cognition has considerably increased over the recent years (see e.g., Bubic et al., 2010;Pezzulo et al., 2007). From this perspective, the instantiation of a simple behavioral paradigm that allows tapping into anticipatory processes as they occur represents a valuable methodological contribution. The anticipatory hand movement data presented by this and other recent studies Dale et al., 2012) suggest that the application of computer mouse tracking to the study of anticipation is a promising approach that could be favorably applied by future studies to investigate anticipatory processes in different contexts. Computer mouse tracking represents a supplementary/alternative method to other methods that allow tracking processes occurring before an expected target such as eye-tracking and brain imaging techniques with high temporal resolution (e.g., electroencephalogra-phy/ magnetoencephalography; EEG/MEG). As a behavioral measure, hand movements cannot substitute direct measures of brain activity.
However, the mouse tracking methodology is a much cheaper and more accessible single tool to study anticipatory processes in real-time than both eye-tracking and EEG/MEG. Moreover, in comparison to eye-movements, which are ballistic and therefore do not provide a genuinely continuous real-time measure, mouse movements provide continuous time-course data on a single trial level (see Freeman et al., 2011;Magnuson, 2005;Spivey, 2007, for a discussion of this). On the other hand, eye-movements have a lower threshold for execution than hand movements, and might therefore be able to capture subtler and more transiently occurring processes than hand movements. Hence, future research would benefit from combining anticipatory hand movements with eye-movements and measures of anticipatory brain activity for a more complete understanding of anticipatory processes.

Footnotes
1 In a pilot study, in which the cues were not displayed as a part of the instruction screen, the majority of the participants reported to have ignored the cues, thinking that they were simply "distractors".
2 That is, having one block with only certain trials, one block with only semi-certainty trials, and one block with only uncertain trials, with the order of certain, semi-certain, and uncertain blocks counterbalanced across participants.

Author note
I am grateful to Bo Sommerlund for his help with programming of this experiment and development of software for data analysis, to Klaus Baerentsen and Risto Näätänen for comments on an earlier version of this manuscript, and to Maria Nordfang for comments on the current version of our article.